Method of estimating parameter of three-dimensional (3D) display device and 3D display device using the method

ABSTRACT

A method of estimating a parameter of a three-dimensional (3D) display device includes estimating a transformation parameter between a camera and a display based on an image displayed on the display and an image reflected by a reflector.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims under 35 U.S.C. § 119 to Korean PatentApplication No. 10-2015-0111744, filed on Aug. 7, 2015, in the KoreanIntellectual Property Office, the entire contents of which areincorporated herein by reference in its entirety.

BACKGROUND

1. Field

At least one example embodiment relates to a method of estimating aparameter of a three-dimensional (3D) display device and/or a 3D displaydevice using the method.

2. Description of the Related Art

Among factors related to three-dimensional (3D) image recognition, theforemost factor is a disparity between images viewed by both eyes of auser. Methods of providing different images to both eyes of a user maybe classified into a stereoscopic type and an autostereoscopic type. Thestereoscopic type method may filter a desired image through divisionmethods that involve using a polarized light, time division, andwavelength division for differentiating a wavelength of a primary color.The autostereoscopic type method may enable images to be viewed only ina desired (or alternatively, predetermined) space using a 3D transformdevice, for example, a parallax barrier, a lenticular lens, or adirectional backlight unit.

The autostereoscopic type method may alleviate an inconvenience ofwearing glasses. In the autostereoscopic method, to restrain crosstalkof a 3D image, the 3D image is to be accurately projected to both eyesof a user. When an error occurs in a process of detecting positions ofboth eyes of a user, or when an error differing from a designed valueoccurs in a process of producing or installing a 3D display device and a3D transform device, the quality of the image may deteriorate.

SUMMARY

At least one example embodiment relates to a method of estimating aparameter of a display device.

In at least one example embodiment, the method may include displaying,on a display, a first image including a first pattern, capturing secondimages generated when the first image is reflected by a reflector atdifferent angles, using a camera combined with the display, andestimating a first transformation parameter between the camera and thedisplay based on the first image and the second images.

The first transformation parameter may include at least one of arotation matrix that performs transformation between coordinates of thecamera and coordinates of the display, and a translation vector thatperforms transformation between coordinates of the camera andcoordinates of the display.

The estimating may include determining the first transformationparameter that minimizes projection errors between the first pattern andvirtual patterns corresponding to the second images.

The estimating may include estimating geometric relationships betweenthe display and the reflector based on the second images, calculatingprojection errors between the first pattern and virtual patternscorresponding to the second images based on the geometric relationships,and updating the first transformation parameter to reduce the projectionerrors.

The method may further include at least one of estimating an intrinsicparameter of the camera based on the first pattern and at least one ofthe second images, and estimating a second transformation parameterbetween the display and an optical layer combined with the display basedon a second pattern included in the first image and at least one of thesecond images.

The estimating of the second transformation parameter may includeestimating the second transformation parameter based on a period of thesecond pattern, a period of a third pattern included in the at least onesecond image, and a gradient of the third pattern.

The method may further include at least one of obtaining the period ofthe second pattern based on a subpixel structure of the display, andobtaining the period of the third pattern and the gradient of the thirdpattern in a Fourier space in which the third pattern isfrequency-transformed.

The intrinsic parameter may include at least one of a focal length ofthe camera, a center position of the camera, and a skewness of thecamera.

The second transformation parameter may include at least one of a pitchof the optical layer and a rotation angle between the optical layer andthe display.

The second pattern may include at least one of a pattern in whichcontinuous lines of an identical brightness are arranged at intervals,and a pattern including areas with an identical color and an identicalbrightness.

The first pattern may include a pattern in which a shape is repeated.Each of the virtual patterns is a virtual image generated when the firstpattern is focused at a corresponding angle in the reflector. The secondpattern may be included in a shape repeated in the first pattern.

At least one example embodiment relates to a three-dimensional (3D)display device.

In at least one example embodiment, the 3D display device may include acamera configured to perform eye tracking, a display configured todisplay a 3D image, and a processor configured to control the display todisplay a first image including a first pattern, control the camera tocapture second images generated when the first image is reflected by areflector at different angles, estimate a first transformation parameterbetween the camera and the display based on the first image and thesecond images, track eyes of the user based on the first transformationparameter, and render the 3D image based on the tracked eyes.

At least one example embodiment relates to a method of estimating aparameter of a display device.

In at least one example embodiment, the method may include displaying,on a display, a first image including a first pattern and a secondpattern, capturing a second image generated when the first image isreflected in a reflector, using a camera combined with the display,estimating a parameter of the camera based on an area corresponding tothe first pattern in the second image, estimating a first transformationparameter between the camera and the display based on the areacorresponding to the first pattern in the second image, and estimating asecond transformation parameter between the display and an optical layercombined with the display based on an area corresponding to the secondpattern in the second image.

At least one example embodiment relates to a display device.

In at least one example embodiment, the display device may include adisplay configured to display a first image including a first patternand a second pattern, a camera combined with the display to capture asecond image generated when the first image is reflected in a reflector,and a processor configured to estimate a parameter of the camera basedon an area corresponding to the first pattern in the second image,estimate a first transformation parameter between the camera and thedisplay based on the area corresponding to the first pattern in thesecond image, and estimate a second transformation parameter between thedisplay and an optical layer combined with the display based on an areacorresponding to the second pattern in the second image.

At least one example embodiment relates to a device including aprocessor and a memory including computer readable instructions, whichwhen executed by the processor, cause the processor to receive a firstimage displayed by a display, the first image including a first pattern.The computer readable instructions cause the processor to receive secondimages, the second images being reflected versions of the first imagethat are captured at different angles. The computer readableinstructions cause the processor to estimate a first transformationparameter between the camera and the display based on the first imageand the second images.

The first transformation parameter includes at least one of a rotationmatrix and a translation vector, and the memory includes computerreadable instructions to cause the processor to transform a coordinatesystem of the camera into a coordinate system of the display based on atleast one of the rotation matrix and the translation vector, and rendera three-dimensional image using the transformed coordinate system.

The processor is configured to estimate by determining the firsttransformation parameter as a transformation parameter that minimizesprojection errors between the first pattern and virtual patternscorresponding to the second images.

Each of the virtual patterns is a virtual image generated when the firstpattern is focused at a corresponding one of the different angles.

The processor is configured to estimate by estimating geometricrelationships between the display and a reflector based on the secondimages, the reflector being used to generate the second images,calculating projection errors between the first pattern and virtualpatterns corresponding to the second images based on the geometricrelationships, and updating the first transformation parameter to reducethe projection errors.

The first image includes a second pattern, and the memory includescomputer readable instructions to cause the processor to estimate anintrinsic parameter of the camera based on the first pattern and atleast one of the second images, and estimate a second transformationparameter between the display and an optical layer associated with thedisplay based on the second pattern and the at least one of the secondimages.

The at least one of the second images includes a third pattern, and theprocessor is configured to estimate the second transformation parameterbased on a period of the second pattern, a period of the third pattern,and a gradient of the third pattern.

The second transformation parameter includes at least one of a pitch ofthe optical layer and a rotation angle between the optical layer and thedisplay, and the memory includes computer readable instructions to causethe processor to render a three-dimensional image based on at least oneof the pitch and the rotation angle.

At least one example embodiment relates to a method including receivinga first image displayed by a display, the first image including a firstpattern. The method includes receiving second images, the second imagesbeing reflected versions of the first image that are captured atdifferent angles. The method includes estimating a first transformationparameter between the camera and the display based on the first imageand the second images, and rendering a three-dimensional image based onthe first transformation parameter.

Additional aspects of example embodiments will be set forth in part inthe description which follows and, in part, will be apparent from thedescription, or may be learned by practice of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects will become apparent and more readilyappreciated from the following description of example embodiments, takenin conjunction with the accompanying drawings of which:

FIG. 1 illustrates a relationship among parameters according to at leastone example embodiment;

FIGS. 2A and 2B illustrate a process of estimating parameters accordingto at least one example embodiment;

FIG. 3 illustrates a method of acquiring a captured image according toat least one example embodiment;

FIG. 4 illustrates types of patterns according to at least one exampleembodiment;

FIG. 5 illustrates a method of acquiring captured images according to atleast one example embodiment;

FIG. 6 illustrates feature points according to at least one exampleembodiment;

FIG. 7 illustrates a process of calculating a first transformationparameter according to at least one example embodiment;

FIG. 8 illustrates coordinates of a virtual pattern according to atleast one example embodiment;

FIG. 9 illustrates restored virtual patterns according to at least oneexample embodiment;

FIG. 10A illustrates geometric relationships among virtual patterns, adisplay, and a reflector according to at least one example embodiment;

FIG. 10B illustrates a process of estimating a first transformationparameter through virtual planes according to at least one exampleembodiment;

FIG. 11 illustrates a process of estimating a first transformationparameter according to at least one example embodiment;

FIG. 12 illustrates a process of estimating a second transformationparameter according to at least one example embodiment;

FIG. 13 illustrates a process of transforming an extracted areaaccording to at least one example embodiment;

FIG. 14 illustrates a process of outputting a second pattern accordingto at least one example embodiment;

FIGS. 15A through 15C illustrate a geometric relationship between animage of a third pattern and a second transformation parameter accordingto at least one example embodiment;

FIG. 16 illustrates a relationship between a number of elements on anoptical layer corresponding to a single period of a second pattern and arotation angle of the optical layer according to at least one exampleembodiment;

FIG. 17 illustrates a scheme of measuring a coefficient corresponding toa gradient of a third pattern and a coefficient corresponding to aperiod of the third pattern by performing a Fourier transform on animage of the third pattern according to at least one example embodiment;

FIGS. 18 and 19 illustrate a process of estimating parameters in a casein which a plurality of cameras is provided according to at least oneexample embodiment;

FIGS. 20 and 21 illustrate a process of estimating parameters of adisplay device including a camera according to at least one exampleembodiment; and

FIG. 22 illustrates an electronic system according to at least oneexample embodiment.

DETAILED DESCRIPTION

Inventive concepts will now be described more fully with reference tothe accompanying drawings, in which example embodiments of are shown.These example embodiments are provided so that this disclosure will bethorough and complete, and will fully convey inventive concepts of tothose skilled in the art. Inventive concepts may be embodied in manydifferent forms with a variety of modifications, and a few embodimentswill be illustrated in drawings and explained in detail. However, thisshould not be construed as being limited to example embodiments setforth herein, and rather, it should be understood that changes may bemade in these example embodiments without departing from the principlesand spirit of inventive concepts, the scope of which are defined in theclaims and their equivalents. Like numbers refer to like elementsthroughout. In the drawings, the thicknesses of layers and regions areexaggerated for clarity.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms. These terms are only used to distinguish oneelement from another. For example, a first element could be termed asecond element, and, similarly, a second element could be termed a firstelement, without departing from the scope of example embodiments. Asused herein, the term “and/or” includes any and all combinations of oneor more of the associated listed items.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element or intervening elements may bepresent. In contrast, when an element is referred to as being “directlyconnected” or “directly coupled” to another element, there are nointervening elements present. Other words used to describe therelationship between elements should be interpreted in a like fashion(e.g., “between” versus “directly between,” “adjacent” versus “directlyadjacent,” etc.).

Unless specifically stated otherwise, or as is apparent from thediscussion, terms such as “processing” or “computing” or “calculating”or “determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical, electronicquantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

Specific details are provided in the following description to provide athorough understanding of example embodiments. However, it will beunderstood by one of ordinary skill in the art that example embodimentsmay be practiced without these specific details. For example, systemsmay be shown in block diagrams so as not to obscure example embodimentsin unnecessary detail. In other instances, well-known processes,structures and techniques may be shown without unnecessary detail inorder to avoid obscuring example embodiments.

Although a flow chart may describe the operations as a sequentialprocess, many of the operations may be performed in parallel,concurrently or simultaneously. In addition, the order of the operationsmay be re-arranged. A process may be terminated when its operations arecompleted, but may also have additional steps not included in thefigure. A process may correspond to a method, function, procedure,subroutine, subprogram, etc. When a process corresponds to a function,its termination may correspond to a return of the function to thecalling function or the main function.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “includes”, “including”,“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

Although corresponding plan views and/or perspective views of somecross-sectional view(s) may not be shown, the cross-sectional view(s) ofdevice structures illustrated herein provide support for a plurality ofdevice structures that extend along two different directions as would beillustrated in a plan view, and/or in three different directions aswould be illustrated in a perspective view. The two different directionsmay or may not be orthogonal to each other. The three differentdirections may include a third direction that may be orthogonal to thetwo different directions. The plurality of device structures may beintegrated in a same electronic device. For example, when a devicestructure (e.g., a memory cell structure or a transistor structure) isillustrated in a cross-sectional view, an electronic device may includea plurality of the device structures (e.g., memory cell structures ortransistor structures), as would be illustrated by a plan view of theelectronic device. The plurality of device structures may be arranged inan array and/or in a two-dimensional pattern.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which inventive concepts belong. It willbe further understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

Expressions such as “at least one of,” when preceding a list ofelements, modify the entire list of elements and do not modify theindividual elements of the list.

The following example embodiments may be applied to a display deviceequipped with a camera, a three-dimensional (3D) display device, and a3D display device equipped with a camera. For example, the exampleembodiments may be used to increase a quality of image in anautostereoscopic 3D display device.

FIG. 1 illustrates a relationship among parameters according to at leastone example embodiment.

Referring to FIG. 1, a 3D display system includes a camera 100, adisplay 200, and a 3D transform device 300. The camera 100 detects aright eye 21 and a left eye 22 of a user. The 3D transform device 300may include an optical layer such as a lens array, a parallax barrier,or a directional backlight unit, for example. The 3D transform device300 divides an image output on the display 200 in to a ray 11corresponding to the right eye 21 and a ray 12 corresponding to the lefteye 22. The ray 11 and the ray 12 are projected to the right eye 21 andthe left eye 22, respectively. The user may view a 3D image through theray 11 and the ray 12 without wearing glasses.

To increase a quality of the 3D image, positions of the right eye 21 andthe left eye 22 are to be detected accurately, and the ray 11 and theray 12 are to be projected accurately to the right eye 21 and the lefteye 22. The quality of the image may be affected by factors such as anerror in a process of transforming a two-dimensional (2D) image to a 3Dimage, and an error differing from a designed value occurring in aprocess of producing and installing the respective devices. Such factorsmay cause crosstalk. When crosstalk occurs, the user may view a blurredimage, or view a distorted image that may cause vertigo if thedistortion is extreme.

The factors may be adjusted by a camera parameter Pc, a pose parameterTcp between the camera 100 and the display 200, and a parameter Pb ofthe 3D transform device. By adjusting such parameters (e.g., forsubsequent image rendering), the quality of the 3D image may increase.

The camera 100 may represent an object in a 3D space as a 2D image.Thus, an error may occur in the process of transforming a 3D space intoa 2D image. For example, the error may be caused by an image sensor or alens in an internal portion of the camera 100. A correspondence betweena 3D space and a 2D image may be estimated through the parameter Pc. Thecamera parameter Pc includes an intrinsic parameter and an extrinsicparameter. A world coordinate system indicating a 3D real space and acamera coordinate system Cc may be transformed (or related) to eachother through the camera parameter Pc. Thus, an error by the camera 100may be compensated for by the camera parameter Pc during imagerendering.

The camera 100 and the display 200 use different coordinate axes. Thepositions of the eyes 21 and 22 of the user estimated through the cameraparameter Pc are based on the camera coordinate system Cc. Thus, thecamera coordinate system Cc is to be transformed to a display coordinatesystem Cp. The pose parameter Tcp between the camera 100 and the display200 indicates a transformation relationship between the cameracoordinate system Cc and the display coordinate system Cp, for example,rotation, and translation. The camera coordinate system Cc may betransformed into the display coordinate system Cp through the poseparameter Tcp. Hereinafter, for ease of description, the pose parameterTcp will be referred to as a first transformation parameter. The 3Ddisplay system detects the eyes 21 and 22 of the user through the camera100, transforms camera coordinates of the eyes 21 and 22 of the userinto display coordinates based on the first transformation parameter,and tracks a viewpoint of the user using the display coordinates.

The rays 11 and 12 output from the display 200 are transferred to theuser through the 3D transform device 300 (or optical layer). Thus,crosstalk may occur due to design errors in the 3D transform device 300.When an error occurs between actual directions of the rays 11 and 12 anddesigned directions, crosstalk may occur. For example, such an error mayinclude a size error and a pose (or position) error. The size error mayrefer to a dimension of an optical layer differing from a designedvalue, and the pose error may refer to a pose (or position) of theoptical layer differing from a designed value. The size error and thepose error may be compensated for through the parameter Pb of the 3Dtransform device 300.

The parameter Pb includes a size and a pose of the 3D transform device300. As a representative example of the size, a pitch of the opticallayer may be used. The pitch of the optical layer may be a distancebetween optical elements. Further, as a representative example of thepose, a rotation angle between the optical layer and a panel may beused. A lens or a barrier included in the optical layer may be slanted.The rotation angle may be an angle at which the lens or the barrierincluded in the optical layer is slanted. Hereinafter, for ease ofdescription, the parameter Pb of the 3D transform device 300 will bereferred to as a second transformation parameter.

In an example, the rotation angle may change based on a slanted angledetermined when the optical layer is manufactured, a height at which theoptical layer is attached to the panel, and an angle at which theoptical layer is attached to the panel. The pose error may becompensated for by the parameter Pb of the 3D transform device 300 andthe pose coefficient cb between the display 200 and the 3D transformdevice 300. In one example, the second transformation parameter mayinclude the parameter Pb of the 3D transform device 300, and the posecoefficient cb between the display 200 and the 3D transform device 300.A three-dimensional image may be rendered using the parameter Pb.

According to at least one example embodiment, by photographing a singleimage or a single set of images, all of the intrinsic parameter of thecamera 100, the first transformation parameter, and the secondtransformation parameter may be obtained.

FIGS. 2A and 2B illustrate a process of estimating parameters accordingto at least one example embodiment. As shown in FIG. 2A, a methodaccording to at least one example embodiment may include operations1010, 1020, 1040, 1045, and/or 1047. As shown in FIG. 2B, a methodaccording to at least one example embodiment may include operations1010, 1020, 1030, 1040, 1050, 1053 and/or 1055. The operations in FIGS.2A and 2B may be carried out and/or caused by processor 2810 from FIG.22.

Referring to FIG. 2A, in operation 1010, a first image is displayed on adisplay. The first image may include a first pattern. In operation 1020,second images generated when the first image is reflected by a reflectorat different angles are captured. The second images may be capturedusing a camera combined with a display. The camera may capture an imagegenerated when an image displayed on the display is reflected in thereflector. When capturing the image reflected in the reflector, settinga z coordinate value of the image displayed on the display to “0” may bereliable. Further, use of the image output on the display to estimate apose between the camera and the display may be useful. A process ofacquiring a captured image through a reflector will be described indetail with reference to FIG. 3. In operation 1040, a firsttransformation parameter is calculated based on an extrinsic parameterof the camera. A process of calculating a first transformation parameterwill be described with reference to FIG. 7. In operation 1045, thecamera coordinate system Cc may be transformed into the displaycoordinate system Cp through the first transformation parameter Tcp. Inoperation 1047, a three-dimensional (3D) image may be rendered based onthe transformed coordinate system. For example, the transformedcoordinate system may be applied to tracking eyes of a user whenrendering a 3D image by the 3D system of FIG. 1 so that a quality of the3D image is enhanced.

FIG. 3 illustrates a method of acquiring a captured image according toat least one example embodiment.

Referring to FIG. 3, the camera 100 may capture a second image 410generated when a first image 210 output on the display 200 is reflectedby a reflector 400. The reflector 400 may include a mirror. When thefirst image 210 is reflected in the reflector 400, the first image 210may appear to be positioned behind the reflector 400. The second image410 focused in the reflector 400 may be a virtual image. The first image210 and the second image 410 include desired (or alternatively,predetermined) patterns for estimating a parameter. Hereinafter, thepattern included in the second image 410 will be referred to as avirtual pattern. Types of patterns will be described in detail withreference to FIG. 4.

FIG. 4 illustrates types of patterns according to at least one exampleembodiment. As described above, a first image and a second image includedesired (or alternatively, predetermined) patterns. A pattern may have arepeated shape for estimating a parameter. For example, the pattern maybe a chess board pattern 210-1, a point arrangement pattern 210-2, aconcentric circle pattern 210-3, and a modified pattern 210-4.Hereinafter, a pattern having a single repeated shape, similar to thepatterns 210-1, 210-2, 210-3, and 210-4, or a pattern having a modifiedsingle repeated shape will be referred to as a first pattern.

The first pattern may include another pattern therein. For example, thefirst pattern may include a pattern 211-1 and a pattern 211-2 therein.Hereinafter, a pattern included in the first pattern will be referred toas a second pattern. The second pattern may be included in an areadistinguished from the first pattern similar to the pattern 211-1, ormay be included in partial areas forming the first pattern similar tothe pattern 211-2. The second pattern may include a pattern includingcontinuous areas of an identical color and an identical brightnesssimilar to the pattern 211-1, and a pattern in which continuous lines ofan identical brightness are disposed at desired (or alternatively,predetermined) intervals similar to the pattern 211-2.

The first pattern may be used to estimate the camera parameter Pc andthe first transformation parameter, and the second pattern may be usedto estimate the second transformation parameter. Thus, when the firstpattern and the second pattern are used simultaneously, the cameraparameter Pc, the first transformation parameter, and the secondtransformation parameter may be obtained by photographing a single imageor a single set of images.

FIG. 5 illustrates a method of acquiring captured images according to atleast one example embodiment.

Referring to FIG. 5, the display outputs a first pattern 210 and asecond pattern 211. The first pattern 210 and the second pattern 211 maybe reflected by the reflector 400. The camera 100 may capture a virtualpattern reflected by the reflector 400. The camera 100 may acquirecaptured images by a movement of the reflector 400. For example, thereflector 400 may rotate and/or move, and the camera 100 may acquireimages captured at various angles and positions. The reflector 400 maybe manipulated through a mechanical machine and/or be manipulatedmanually by a user. The number of angles and positions may be a designparameter selected based on empirical evidence. Although not shown inthe drawings, when the display 200 and the camera 100 are included in amobile device, the display 200 and the camera 100 may move or rotate infront of the fixed reflector 400 to acquire the captured images.

FIG. 6 illustrates feature points according to at least one exampleembodiment.

Referring to FIG. 6, the first pattern 210, the second pattern 211, anda feature point 212 of the first pattern 210 are illustrated. The firstpattern 210 may include the feature point 212. The feature point 212 maybe positioned at a center of a circle or at an intersecting point of apattern. The feature point 212 may be detected using corner detectionand/or circle detection. The feature point 212 may be used to estimate aparameter. When the feature point 212 is used, an amount of time used toestimate a parameter may be reduced.

Hereinafter, a process of estimating parameters will be described indetail with reference to FIG. 2B.

In operation 1030, the camera parameter Pc may be calculated based onthe captured images. As described above, the camera parameter Pcincludes an intrinsic parameter and an extrinsic parameter. Theintrinsic parameter includes a focal length, a center position, and/or askewness of the camera 100. The extrinsic parameter includes a rotationparameter, and/or a translation parameter. The intrinsic parameter maybe expressed by Equation 1.

$\begin{matrix}{K = \begin{bmatrix}f_{x} & s & u_{c} \\0 & f_{y} & v_{c} \\0 & 0 & 1\end{bmatrix}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

In Equation 1, K denotes an intrinsic parameter, fx denotes a focallength with respect to an axis x of a camera, and fy denotes a focallength with respect to an axis y of the camera. uc and vc denotecoordinates x and y at which a principle axis of the camera 100 or anaxis z of camera coordinates meets an image plane. s denotes a gradientof a pixel. A relationship between a point X of a world coordinatesystem and a point x projected on the image plane may be expressed byEquation 2.x=K[Rt]X  [Equation 2]

In Equation 2, R denotes a rotation parameter, and t denotes atranslation parameter. Equation 2 may be expressed by Equation 3 andEquation 4.

$\begin{matrix}{x = {f\left( {K,R,t,X} \right)}} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack \\{X = {{\begin{bmatrix}f_{x} & s & u_{c} \\0 & f_{y} & v_{c} \\0 & 0 & 1\end{bmatrix}\begin{bmatrix}r_{1} & r_{2} & r_{3} & r_{1} \\r_{4} & r_{5} & r_{6} & r_{2} \\r_{7} & r_{8} & r_{9} & r_{3}\end{bmatrix}}X}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

In Equation 3, when a pattern is captured n times, a j-th correspondingpoint, among m corresponding points with respect to a feature point inan i-th captured image, may be expressed by Equation 5. Here, thecorresponding point refers to a point corresponding to an image capturedby the camera 100 with respect to a feature point 212 of a patternoutput on a display.x _(j) =f(K,R _(i) ,t _(i) ,X _(j))  [Equation 5]

The camera parameter Pc that reduces (or alternatively, minimizes) aprojection error when all of m corresponding points of the n capturedimages are projected may be estimated through Equation 6.

$\begin{matrix}{{argmin}{\sum\limits_{i = 1}^{n}{\sum\limits_{j = 1}^{m}{{x_{j} - {f\left( {K,R_{i},t_{i},X_{j}} \right)}}}^{2}}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

In Equation 6, f(X) denotes a value obtained by projecting X, and may beexpressed by [u_(d) v_(d) 1]. In Equation 6, f(X) includes a distortion.The camera parameter Pc may include a radial distortion. The accuracy ofthe camera parameter Pc may increase through the radial distortion. Inview of the radial distortion, Equation 6 may be expressed by Equation7.

$\begin{matrix}{{argmin}{\sum\limits_{i = 1}^{n}{\sum\limits_{j = 1}^{m}{{x_{j} - {f^{\prime}\left( {K,k,R_{i},t_{i},X_{j}} \right)}}}^{2}}}} & \left\lbrack {{Equation}\mspace{14mu} 7} \right\rbrack\end{matrix}$

In Equation 7, k denotes a radial distortion. f′(X) of Equation 7 may beexpressed by [u_(u) v_(u) 1]. In view of k, relational expressions amongk, r, u_(u), and v_(u) may be expressed by Equation 8. To reduce acomputational complexity, when distortion of a camera lens is relativelysmall, the camera parameter Pc may be obtained through Equation 6.

$\begin{matrix}{{k = \left\{ {k_{1},k_{2},k_{3},k_{4},k_{5}} \right\}}{r = \sqrt{\left( {u_{d} - u_{c}} \right)^{2} + \left( {v_{d} - v_{c}} \right)^{2}}}{u_{u} = \frac{u_{d}}{1 + {k_{1}r^{2}} + {k_{2}r^{4}} + {k_{3}r^{6}} + {k_{4}r^{8}} + {k_{5}r^{10}}}}{v_{u} = \frac{v_{d}}{1 + {k_{1}r^{2}} + {k_{2}r^{4}} + {k_{3}r^{6}} + {k_{4}r^{8}} + {k_{5}r^{10}}}}} & \left\lbrack {{Equation}\mspace{14mu} 8} \right\rbrack\end{matrix}$

In operation 1040, a first transformation parameter may be calculatedbased on the extrinsic parameter of the camera. A process of calculatinga first transformation parameter will be described with reference toFIG. 7.

FIG. 7 illustrates a process of calculating a first transformationparameter according to at least one example embodiment. A methodaccording to at least one example embodiment may include one or more ofoperations 1210-1230. The operations in FIG. 7 may be carried out and/orcaused by processor 2810 from FIG. 22.

Referring to FIG. 7, in operation 1210, positions of virtual patternscorresponding to second images are determined. The positions of thevirtual patterns may be determined through Equation 9.

$\begin{matrix}{X_{ij} = {{\begin{bmatrix}R_{i} & t_{i} \\0 & 1\end{bmatrix}\begin{bmatrix}I & t_{j} \\0 & 1\end{bmatrix}}\begin{bmatrix}0 \\0 \\0 \\1\end{bmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 9} \right\rbrack\end{matrix}$

In Equation 9, x_(ij) denotes a position of a j-th corresponding pointin an i-th image. R_(i) and t_(i) denote a rotation parameter and atranslation parameter in the i0th image, respectively. The rotationparameter and the translation parameter may be determined from theextrinsic parameter of the camera. t_(j) denotes coordinates of a j-thfeature point. t_(j) will be described in detail with reference to FIG.8.

FIG. 8 illustrates coordinates of a virtual pattern according to atleast one example embodiment.

Referring to FIG. 8, a coordinate axis 213 of a pattern, and positionsof four points with respect to the coordinate axis 213 are illustrated.In FIG. 8, d denotes a length of a single side of a rectangular includedin a chess pattern. As shown in FIG. 8, tj may be a 1×3 matrixindicating coordinates of a j-th corresponding point.

Referring to FIG. 7 again, virtual patterns corresponding to n imagesmay be restored through Equation 9. FIG. 9 illustrates restored virtualpatterns according to at least one example embodiment.

In operation 1220, geometric relationships among the virtual patterns,the display, and the reflector are estimated. Operation 1220 will bedescribed in detail with reference to FIG. 10A.

FIG. 10A illustrates geometric relationships among virtual patterns, adisplay, and a reflector according to at least one example embodiment.

Referring to FIG. 10A, a geometric relationship among an estimatedvirtual pattern 405, the display 200, and the reflector 400 isillustrated. A position of the display 200 may be determined based on aninitial value of the first transformation parameter. The initial valueof the first transformation parameter may be set to be a designed valueor a desired (or alternatively, predetermined) value. The firsttransformation parameter may be expressed by Equation 10.

$\begin{matrix}{{T_{cp}\begin{bmatrix}R_{cp} & t_{cp} \\0 & 1\end{bmatrix}} = \begin{bmatrix}r_{{{cp}\_}11} & r_{{{cp}\_}12} & r_{{{cp}\_}13} & r_{{cp}\_ x} \\r_{{{cp}\_}21} & r_{{{cp}\_}22} & r_{{{cp}\_}23} & r_{{cp}\_ y} \\r_{{{cp}\_}31} & r_{{{cp}\_}32} & r_{{{cp}\_}33} & r_{{cp}\_ z} \\0 & 0 & 0 & 1\end{bmatrix}} & \left\lbrack {{Equation}\mspace{14mu} 10} \right\rbrack\end{matrix}$

In Equation 10, T_(cp) denotes a first transformation parameter, R_(cp)denotes a rotation parameter of a display coordinate system Cp withrespect to a camera coordinate system Cc, and tcp denotes a translationparameter of the display coordinate system Cp with respect to the cameracoordinate system Cc.

When the position of the display 200 is set by setting the initial valueof the first transformation parameter, a feature point of a firstpattern may be projected as the virtual pattern 405. When a differencebetween the first transformation parameter and a true value decreases, adistance between the feature point projected based on the firsttransformation parameter and the corresponding point of the virtualpattern 405 decreases. Conversely, when the difference between the firsttransformation parameter and the true value increases, the distancebetween the feature point projected based on the first transformationparameter and the corresponding point of the virtual pattern 405increases.

Hereinafter, a method of projecting the feature point of the display 200set by the first transformation parameter as the virtual pattern 405will be described. A j-th feature point of the display 200 may beexpressed by Equation 11.

$\begin{matrix}{{\overset{\_}{X}}_{j} = {{\begin{bmatrix}R_{cp} & t_{cp} \\0 & 1\end{bmatrix}\begin{bmatrix}I & t_{j} \\0 & 1\end{bmatrix}}\begin{bmatrix}0 \\0 \\0 \\1\end{bmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 11} \right\rbrack\end{matrix}$

In Equation 11, X _(j) denotes a j-th feature point of the display 200,and tj denotes a translated position of the j-th feature point withrespect to the origin of the display 200. The remaining parameters arethe same as expressed by Equation 10. Since all feature points arepresent on the same plane, the feature points have identical normalvectors. A normal vector n of the display 200 corresponds to a third rowof Rcp, and may be expressed by Equation 12.

$\begin{matrix}{\overset{\_}{n} = {R_{cp}\begin{bmatrix}0 \\0 \\1\end{bmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack\end{matrix}$

A pose between the virtual pattern 405 and the camera 100 are knownthrough the extrinsic parameter of the camera 100. Thus, positions ofthe camera 100 and the virtual pattern 405 may be specified. Theposition of the display 200 may be set based on the initial value of thefirst transformation parameter, and the position of the reflector 400may be set at a midpoint between the display 200 and the virtual pattern405. An angle formed by the reflector 400 and the display 200 isidentical to an angle formed by the reflector 400 and the virtualpattern 405. Thus, using the aforementioned method, the geometricrelationships among the virtual pattern 405, the display 200, and thereflector 400 may be estimated.

Referring to FIG. 7 again, in operation 1230, projection errors betweenthe first pattern and the virtual patterns are reduced (oralternatively, minimized). A process of projecting a feature point and aprocess of calculating a projection error of the feature point will bedescribed with reference to FIG. 10A. A normalized normal vector n_(mi)of the display 200 may be expressed by Equation 13.

$\begin{matrix}{n_{mi} = \frac{\overset{\_}{n} + n_{i}}{{\overset{\_}{n} + n_{i}}}} & \left\lbrack {{Equation}\mspace{14mu} 13} \right\rbrack\end{matrix}$

In Equation 13, n denotes a normal vector of the display 200, and n_(i)denotes a normal vector of the virtual pattern 405. As shown in FIG. 6,when the feature point X _(j) of the display 200 is translated by d_(ij)in a normal direction n_(mi) of the reflector 400, the feature point X_(j) is projected at a position of X_(ij). d_(ij) may be expressed byEquation 14.d _(ij) =∥X _(j) −X _(ij)∥  [Equation 14]

Further, projection of the aforementioned feature point may be expressedby Equation 15.T _(ij)( X _(J))= X _(J) +d _(ij) n _(mi)  [Equation 15]

The projection error indicates a Euclidian distance between t_(ij)(X_(j)) to which the feature point X _(j) of the set display 200 isprojected and a feature point X_(ij) of a corresponding i-th virtualpattern. An average Em of the error occurring when the feature point X_(j) is projected to all virtual patterns may be expressed by Equation16.

$\begin{matrix}{E_{m} = {\frac{1}{MN}{\sum\limits_{i = 1}^{N}{\sum\limits_{j = 1}^{M}{{X_{ij} - {T_{ij}\left( {\overset{\_}{X}}_{j} \right)}}}}}}} & \left\lbrack {{Equation}\mspace{14mu} 16} \right\rbrack\end{matrix}$

When a value of Em that is obtained by changing the initial value of thefirst transformation parameter to a desired (or alternatively,predetermined) value is less than Em obtained by the initial value, thefirst transformation parameter may be updated to the changed value. Byiterating such a process until the value of the first transformationparameter does not change, a final value of the first transformationparameter may be obtained.

FIG. 10B illustrates a process of estimating a first transformationparameter through virtual planes according to at least one exampleembodiment. In FIG. 10B, a first transformation parameter that reduces(or alternatively, minimizes) errors between {tilde over (X)}_(j) andX_(1J) , X_(2J) , and X_(3J) obtained based on the display 200,reflectors 401, 402, and 403, and virtual planes 411, 412, and 413 maybe determined to be the final first transformation parameter.

FIG. 11 illustrates a process of estimating a first transformationparameter according to at least one example embodiment. As shown in FIG.11, a method according to at least one example embodiment may includeone or more of operations 1310-1410. The operations in FIG. 11 may becarried out and/or caused by processor 2810 from FIG. 22.

Referring to FIG. 11, in operation 1310, coordinates of a virtualpattern are calculated based on a rotation R and a translation t of thevirtual pattern. The rotation R and the translation t of the virtualpattern may be obtained from an extrinsic parameter of camera. Inoperation 1320, a first transformation parameter Tcp is set. In aninitial iteration, the first transformation parameter Tcp may be set tobe an initial Tcp and an index i, indicating an image of a virtualpattern, is set to an initial value (e.g., a value of 1). As iterationsare performed, the first transformation parameter Tcp may be set to amodified first transformation parameter Tcp.

In operation 1325, the index i, indicating an image of a virtualpattern, is incremented (i=i+1).

In operation 1328, it is determined whether i+1 is greater than a numbern. If so, the method proceeds to operation 1370 (described below). Ifnot, the method proceeds to operation 1330. The number n may be a numberof virtual patterns on which operations 1330-1360 are performed.

In operation 1330, a normal vector n_(mi) corresponding to each virtualpattern is set using the index i indicating an image of a virtualpattern.

In operation 1335, an index j, indicating a feature point in eachvirtual pattern, is incremented (j=j+1, where j has an initial value of,for example, 1).

In operation 1338, it is determined whether j+1 is greater than a numberm. If so, the method returns to operation 1325 to increment i and searchfor a next virtual pattern. If not, the method proceeds to operation1340. The number m may be a number of feature points on which operation1340 and 1360.

In operation 1340, a distance between feature points is calculated usingthe index j indicating a feature point in each virtual pattern. Forexample, a distance between a feature point displayed on a display and afeature point in a virtual pattern may be calculated. In operation 1360,a projection error Em is calculated. For example, the projection errorEm may be calculated based on a cumulative sum of distances betweenfeature points calculated with respect to the virtual patterns. Themethod may return to operation 1335 to increment j and search foranother feature point in the virtual pattern.

When the projection error Em is calculated for all virtual patterns(i.e., when i+1>n occurs), whether the projection error Em is less thana previous projection error Ep is verified in operation 1370. Theprevious projection error Ep is a projection error calculated in aprevious iteration, and may be set to be a sufficiently great value inan initial iteration.

When the projection error Em is less than the previous projection errorEp, the previous projection error Ep is updated in operation 1380, andthe first transformation parameter Tcp is modified in operation 1390 andthe method returns to operation 1320 (where i is reset to an initialvalue, e.g., 1). When the projection error Em is greater than or equalto the previous projection error Ep, whether a difference between theprevious projection error Ep and the projection error Em is less than athreshold ε is verified in operation 1400. The threshold ε may be adesign parameter set based on empirical evidence.

When the difference between the previous projection error Ep and theprojection error Em is greater than or equal to the threshold ε, thefirst transformation parameter Tcp may be modified in operation 1390.Conversely, when the difference between the previous projection error Epand the projection error Em is less than the threshold ε, the firsttransformation parameter Tcp is confirmed in operation 1410. Theconfirmed first transformation parameter Tcp may be the firsttransformation parameter Tcp modified in the previous iteration.

Referring to FIG. 2B again, in operation 1050, a second transformationparameter may be calculated based on the captured images. A process ofcalculating a second transformation parameter will be described indetail with reference to FIGS. 12 through 17. In operation 1053, thefirst transformation parameter Tcp is used to transform the cameracoordinate system Cc into the display coordinate system Cp. In operation1055, a 3D image may be rendered based on the transformed coordinatesystem, the second transformation parameter Pb and the camera parameterPc. For example, the transformed coordinate system may be used to trackeyes of the user while the second transformation parameter Pb and thecamera parameter Pc may be applied to rendering a 3D image by the 3Dsystem of FIG. 1 so that a quality of the 3D image is enhanced.

FIG. 12 illustrates a process of estimating a second transformationparameter according to at least one example embodiment. As shown in FIG.12, a method according to at least one example embodiment may includeone or more of operations 2200-2250. The operations in FIG. 12 may becarried out and/or caused by processor 2810 from FIG. 22.

Referring to FIG. 12, in operation 2200, an index i, indicating avirtual pattern, is incremented by 1 (e.g., from an initial value of 1).

In operation 2205, it is determined whether i+1 is greater than a numbern. The number n may be a number of virtual patterns on which operations2210-2240 are performed.

In operation 2210, an area corresponding to a second pattern isextracted from each virtual pattern using the index i indicating avirtual pattern. The second pattern displayed on a display may be areasof an identical color and an identical brightness, for example, a whitebackground. In an example, in a case in which a slanted barrier is usedas an optical layer, an image photographed through a mirror when thewhite background is displayed on a monitor includes a patterncorresponding to the slanted barrier. Referring to a pattern 210-5 andthe pattern 211-1 of FIG. 4, the pattern 210-5 may be used to estimatethe camera parameter and the first transformation parameter, and thepattern 211-1 may be used to estimate the second transformationparameter.

In another example, when a slanted lens is used as the optical layer, adirect observation of the lens may be impossible. Thus, a pattern inwhich continuous lines of an identical brightness are disposed atdesired (or alternatively, predetermined) intervals, for example, asingle-colored stripe pattern, may be used as a second pattern displayedon the display. Referring to a pattern 210-6 and the pattern 211-2 ofFIG. 4, the pattern 210-6 may be used to estimate the camera parameterand the first transformation parameter, and the pattern 211-2 may beused to estimate the second transformation parameter.

In operation 2220, the extracted area is transformed. The area extractedto estimate the second transformed parameter is a projected area andthus, has a size and shape differing from the original size and shape.Accordingly, a process of transforming the extracted area to have theoriginal size and shape is required. FIG. 13 illustrates a process oftransforming an extracted area according to at least one exampleembodiment. As shown in FIG. 13, a method according to at least oneexample embodiment may include one or more of operations 2310-2340. Theoperations in FIG. 13 may be carried out and/or caused by processor 2810from FIG. 22.

Referring to FIG. 13, since a captured image is an image reflectedthrough a mirror, left and right of the captured image is reversed inoperation 2310. In operation 2320, an area corresponding to a secondpattern is extracted from the left-and-right reversed image, and cornersof the extracted area are detected. Corner points of the extracted areamay be defined as p1, p2, p3, and p4. Here, p1 may correspond to anupper left corner, p2 may correspond to an upper right corner, p3 maycorrespond to a lower left corner, and p4 may correspond to a lowerright corner.

The corner points may have a relationship as expressed by Equation 17.

$\begin{matrix}{\begin{bmatrix}0 & w & 0 & w \\0 & 0 & h & h \\1 & 1 & 1 & 1\end{bmatrix} = {H\begin{bmatrix}p_{1x} & p_{2x} & p_{3x} & p_{4x} \\p_{1y} & p_{2y} & p_{3y} & p_{4y} \\1 & 1 & 1 & 1\end{bmatrix}}} & \left\lbrack {{Equation}\mspace{14mu} 17} \right\rbrack\end{matrix}$

In Equation 17, w denotes an actual width length of a second pattern,and h denotes an actual height length of the second pattern. p_(nx)denotes an coordinate x of p_(n), and p_(ny) denotes a coordinate y ofp_(n). H[ ] denotes a homographic operator.

In operation 2330, a homograph H satisfying Equation 17 is calculated.In operation 2340, the area is transformed by applying H to all pixelsp_(i) belonging to the extracted area S. The area transformation may bereferred to as area warping. The warped pixel p_(i)′ may be expressed byEquation 18.p′ _(i)(iϵS)=Hp _(i)(iϵS)  [Equation 18]

Referring to FIG. 12 again, in operation 2230, a gradient of a patternincluded in the transformed area is calculated. In operation 2240, apitch of the pattern included in the transformed area is calculated. Ina case in which the slanted barrier is used as the optical layer and thewhite background is used as the second pattern, the gradient and thepitch may be deduced directly from the transformed image. For example, aportion indicated in white in the transformed image may correspond to aslit of the optical layer. Thus, by measuring a gradient of a patternindicated in white, an angle at which the optical layer is slanted maybe obtained.

When a distance between slits is defined as ph (pixel), a pitch p of theoptical layer may be calculated as expressed by Equation 19.p=s_(p)P_(h)cos θ  [Equation 19]

In Equation 19, s_(p) denotes a size of a pixel.

When a contrast of the transformed image is relatively low, or when alot of noise is included in the transformed image, it is difficult toextract the second transformation parameter directly from thetransformed image. In this example, the transformed image may betransformed to a frequency domain, and the second transformationparameter may be extracted using Equations 20 and 21.

$\begin{matrix}{\theta = {\frac{\pi}{2} - {\delta.}}} & \left\lbrack {{Equation}\mspace{14mu} 20} \right\rbrack\end{matrix}$

In Equation 20, δ denotes an angle between a vertical line and a lineconnecting points with highest intensities in a frequency domain image,and θ denotes an angle at which an optical layer is slanted.

$\begin{matrix}{p = {\frac{s_{p}w}{\rho}\cos\;\theta}} & \left\lbrack {{Equation}\mspace{14mu} 21} \right\rbrack\end{matrix}$

In Equation 21, ρ denotes a horizontal distance between points havinghighest intensities in a frequency domain image, and p denotes a pitchof an optical layer. After operation 2240, the method returns tooperation 2200 to increment i and search for another virtual pattern onwhich to perform operations 2210-2240.

When gradients and pitches in all virtual patterns are calculated (i.e.,when i+1>n occurs), an average of the gradients and an average of thepitches are calculated in operation 2250.

Hereinafter, an operation of calculating a second transformationparameter in a case in which a slanted lens is used as an optical layerand a second pattern includes a stripe pattern will be described withreference to FIGS. 14 through 17.

FIG. 14 illustrates a process of outputting a second pattern accordingto at least one example embodiment.

Referring to FIG. 14, an image 1421 of a second pattern may correspondto a stripe pattern. The image 1421 of the second pattern may passthrough an optical layer 1423 and be displayed as an image 1430 of athird pattern. The image 1430 of the third pattern may be a ray imagepassing through a central point of the optical layer 1423, for example,a central axis of a lens, or a slit. The image 1430 of the third patternmay include a pattern in which lines including a plurality of points aredisposed at desired (or alternatively, predetermined) intervals. Forexample, the image 1430 of the third pattern may include repetitivelines having a single principal direction.

A slope of each line included in the image 1430 of the third pattern maydiffer from a slope of each line included in the image 1421 of thesecond pattern. For example, lines included in the image 1421 of thesecond pattern may be vertical lines, and lines included in the image1430 of the third pattern may be inclined lines. Further, an intervalbetween the lines included in the image 1430 of the third pattern maydiffer from an interval between the lines included in the image 1421 ofthe second pattern.

As will be described in detail below, a processor may determine thesecond transformation parameter for a 3D display device 1420 byanalyzing two adjacent lines 1431 and 1432 from among the lines includedin the image 1430 of the third pattern. For example, the processor maydetermine the second transformation parameter based on an intervalbetween the line 1431 and the line 1432, and slopes of the line 1431 andthe line 1432. As described above, the second transformation parametermay include a pose and a pitch of the optical layer 1423.

FIGS. 15A through 15C illustrate a geometric relationship between animage of a third pattern and a second transformation parameter accordingto at least one example embodiment.

Referring to FIG. 15A, an image of a third pattern may include aplurality of points constituting lines 1731 and 1732. A processor maymeasure a coefficient α corresponding to a gradient of the third patternfrom the image of the third pattern. For example, the coefficient αcorresponding to the gradient of the third pattern may correspond to aslope of the line 1732. Further, the processor (e.g., processor 2810 inFIG. 22) may measure a coefficient c corresponding to a period of thethird pattern from the image of the third pattern. For example, thecoefficient c corresponding to the period of the third pattern maycorrespond to a vertical interval between the lines 1731 and 1732.

A line 1711 and a line 1712 are lines included in an image of a secondpattern. The line 1711 and the line 1712 are perpendicular to a line1750. The processor may be aware of information on a panel of a 3Ddisplay device in advance. For example, the processor may be aware of asubpixel structure of the panel, a resolution of the panel, and a sizeof the panel. The subpixel structure of the panel may include aninterval between subpixels of a color used for the second pattern in thepanel. The processor may obtain a coefficient g corresponding to aperiod of the second pattern based on the information on the panel. Forexample, the coefficient g corresponding to the period of the secondpattern may be an interval between the lines 1711 and 1712.

The image of the second pattern may be displayed on the panel, and theimage of the third pattern may be captured from a desired (oralternatively, predetermined) point of view. Thus, an actual intervalbetween lines displayed on the panel may differ from the intervalbetween the lines 1711 and 1712 virtually shown on the image of thethird pattern. The processor may obtain the interval between the lines1711 and 1712 based on a ratio of the size of the panel to a size of theimage of the third pattern.

The processor may determine a second transformation parameter for the 3Ddisplay device based on the second pattern and the third pattern. Forexample, the processor may determine the second transformation parameterbased on the coefficient α corresponding to the gradient of the thirdpattern, the coefficient c corresponding to the period of the thirdpattern, and the coefficient g corresponding to the period of the secondpattern. The second transformation parameter may include a coefficient passociated with a dimension of the optical layer and a coefficient θassociated with a pose of the optical layer.

The coefficient p associated with the dimension of the optical layer maybe a pitch of the optical layer. The pitch of the optical layer may bean interval between elements included in the optical layer. For example,the pitch of the optical layer may be a shortest interval between acentral axis 1721 of a first lens and a central axis 1722 of a secondlens. The coefficient θ associated with the pose of the optical layermay be a rotation angle of the optical layer. The rotation angle of theoptical layer may be an angle at which the optical layer rotates withrespect to the panel. For example, the rotation angle of the opticallayer may be an angle between a central axis 1723 of a third lens andthe line 1712 displayed on the panel.

Referring to FIG. 15B, a horizontal interval between the central axis1721 of the first lens and the central axis 1722 of the second lens maybe p/cos θ. p/cos θ may be a horizontal pitch of the optical layer. Thehorizontal interval between the central axis 1721 of the first lens andthe central axis 1722 of the second lens may be equal to a horizontalinterval between the central axis 1722 of the second lens and thecentral axis 1723 of the third lens. Thus, a horizontal interval betweenthe central axis 1721 of the first lens and the central axis 1723 of thethird lens may be 2·p/cos θ.

Referring to a right triangle 1760, a length of a bottom side may be2·p/cos θ−g, a height may be g·tan α, and an acute angle may be θ at theintersection of central axis 1723, line 1712, and line 1732. Based onlengths of two sides and the acute angle of the right triangle 1760,Equation 22 may be deduced.

$\begin{matrix}{{\tan\mspace{11mu}\theta} = \frac{\frac{np}{\cos\mspace{11mu}\theta} - g}{g\mspace{14mu}\tan\mspace{14mu}\alpha}} & \left\lbrack {{Equation}\mspace{14mu} 22} \right\rbrack\end{matrix}$

Equation 22 may be arranged as expressed by Equation 23.g tan α sin θ+g cos θ−np=0  [Equation 23]

In Equation 23, g denotes a coefficient corresponding to a period of asecond pattern, and a denotes a coefficient corresponding to a gradientof a third pattern. θ denotes a rotation angle between an optical layerand a panel, and p denotes a pitch of the optical layer. n denotes anumber of elements of the optical layer corresponding to a single periodof the second pattern. For example, n may correspond to a number oflenses provided between adjacent lines included in an image of the thirdpattern.

Referring to FIG. 15C, a vertical interval between the central axis 1721of the first lens and the central axis 1722 of the second lens may bep/sin θ. Referring to a right triangle 910, a length of a bottom sidemay be p, and one acute angle may be θ. Based on lengths of two sidesand one acute angle of the right triangle 910, Equation 24 may bededuced.

$\begin{matrix}{c = \frac{p}{\sin\mspace{11mu}\theta}} & \left\lbrack {{Equation}\mspace{14mu} 24} \right\rbrack\end{matrix}$

In Equation 24, c denotes a coefficient corresponding to a period of thethird pattern, p denotes the pitch of the optical layer, and θ denotesthe rotation angle between the optical layer and the panel. Theprocessor may determine the pitch p of the optical layer and therotation angle θ between the optical layer and the panel based onEquations 23 and 24.

For ease of description, FIGS. 15A through 15C are described assuming acase in which n corresponds to “2.” However, the pitch p of the opticallayer and the rotation angle θ between the optical layer and the panelmay vary depending on n. In an example, in a situation in which a secondpattern and a third pattern are given, a pitch p of an optical layer anda rotation angle θ between the optical layer and a panel satisfyingEquations 23 and 24 in a case of n being “1” may be calculated. Inanother example, in a situation in which a second pattern and a thirdpattern are given, a pitch p of an optical layer and a rotation angle θbetween the optical layer and a panel satisfying Equations 23 and 24 ina case of n being “3” may be calculated.

In Equations 23 and 24, there are a total of three unknowns n, p, and θ.Since a number of the unknowns is greater than a number of equations, aplurality of solutions satisfying Equations 23 and 24 may exist. Forexample, a relationship between the number n of the elements of theoptical layer corresponding to the single period of the second patternand the rotation angle θ of the optical layer may be represented asshown in a graph of FIG. 16.

The number n of the elements of the optical layer corresponding to thesingle period of the second pattern may be a positive integer greaterthan or equal to “1”, and the rotation angle θ of the optical layer maybe greater than or equal to −90 degrees and less than or equal to +90degrees. Thus, candidate solutions may be extracted from the graph ofFIG. 16. For example, when n is “1”, “2”, “3”, or “4”, θ may be 23.2735degrees, 11.9920 degrees, 8.0214 degrees, or 6.0218 degrees,respectively. When θ is known, p may be calculated based on Equation 24.

When an initial parameter of a 3D display device is known, an optimalsolution may be selected from candidate solutions based on the initialparameter. The initial parameter may be a design value for the 3Ddisplay device. For example, when an optical layer of the 3D displaydevice is designed to have a pitch of 0.5 millimeters (mm), a candidatesolution of n=2 having a pitch p most approximate to the design value of0.5 mm may be finally selected from the candidate solutions. When theoptical layer of the 3D display device is designed to have a rotationangle of 12 degrees, a candidate solution of n=2 having a rotation angleθ most approximate to the design value of 12 degrees may be finallyselected from the candidate solutions. At least one example embodimentmay provide technology that determines a rotation angle of an opticallayer and a pitch of the optical layer simultaneously.

FIG. 17 illustrates a scheme of measuring a coefficient corresponding toa gradient of a third pattern and a coefficient corresponding to aperiod of the third pattern by performing a Fourier transform on animage of the third pattern according to at least one example embodiment.At least one example embodiment may provide technology that measurescoefficients resistant against noise, irrespective of a gradient of acaptured image.

A pattern included in an image of a third pattern may include noise, ormay be irregular. Thus, an error may occur when measuring a gradient ofthe third pattern and/or an interval of the third pattern directly fromthe image of the third pattern. At least one example embodiment mayprovide technology that accurately measures a coefficient correspondingto the gradient of the third pattern and a coefficient corresponding tothe period of the third pattern in a frequency domain by performing aFourier transform on the image of the third pattern.

Referring to FIG. 17, an image 2120 may be a result of performing aFourier transform on an image 2110 of a third pattern. Unlike the image2110 of the third pattern, a gradient and a period of a pattern may beexplicitly represented in the image 2120. Although noise is included inthe image 2110 of the third pattern, the identical gradient and theidentical period may be represented in the image 2120.

A processor (e.g., processor 2810 in FIG. 22) may measure a coefficientcorresponding to the gradient of the third pattern and a coefficientcorresponding to the period of the third pattern using the image 2120. Avector b may be a line connecting points horizontally close to a centerof the image 2120. The vector b may represent a vertical frequencycomponent in the image of the third pattern, and indicate that the imageof the third pattern is inclined. The processor may calibrate remaininggradients based on the vector b. A vector a may be a line connectingpoints having highest intensities in the image 2120. A slope of thevector a may be perpendicular to the gradient of the third pattern. Animage 2130 illustrates an intensity of a partial area in the image 2120.An axis x and an axis y of the image 2130 corresponds to an axis x andan axis y of the image 2120, and a value of an axis z of the image 2130may be represented as a brightness in the image 2120.

The processor may measure the coefficient corresponding to the gradientof the third pattern using Equation 25.

$\begin{matrix}{\alpha = {b - \frac{\pi}{2} - \alpha}} & \left\lbrack {{Equation}\mspace{14mu} 25} \right\rbrack\end{matrix}$

In Equation 25, α denotes the coefficient corresponding to the gradientof the third pattern, b denotes an angle between an axis x of the image2120 and the vector a, and a denotes an angle between the axis x of theimage 2120 and the vector b.

The processor may calculate a number of lines having an identical slopeor an identical principal direction in the image of the third pattern bydividing the overall height of the image 2120 by a height. Here theheight may be a difference in height between points having highestintensities in the image 2120, or a height of a point closest to theorigin of the image 2120, among the points having the highestintensities in the image 2120.

The processor may calculate the coefficient corresponding to the periodof the third pattern by dividing a height of an image of a secondpattern displayed on the 3D display device by the calculated number. Inthis example, the processor may calculate a coefficient corresponding toa period of the second pattern to be an actual interval between lines ofthe second pattern displayed on a panel.

The processor may calculate the coefficient corresponding to the periodof the third pattern by dividing a height of a captured image of a thirdpattern. In this example, the processor may calculate the coefficientcorresponding to the period of the second pattern by adjusting an actualinterval between the lines of the second pattern displayed on the panelbased on a ratio of a size of the panel to a size of the image of thethird pattern.

The processor may determine the second transformation parameter for the3D display device more accurately by iteratively performing theforegoing processes. For example, the processor may determine a finalsecond transformation parameter using statistics of secondtransformation parameters deduced as a result of the iterativelyperforming. In another example, the processor may determine the finalsecond transformation parameter by excluding a second transformationparameter out of a standard distribution from the second transformationparameters. The processor may determine the final second transformationparameter based on the second transformation parameters deduced as aresult of the iteratively performing, thereby reducing (oralternatively, minimizing) a level of the final transformation parameterbeing inaccurate due to an error that may be included in a result of asingle iteration.

FIGS. 18 and 19 illustrate a process of estimating parameters in a casein which a plurality of cameras is provided according to at least oneexample embodiment. As shown in FIG. 19, a method according to at leastone example embodiment may include one or more of operations 2510-2540.The operations in FIGS. 18 and 19 may be carried out and/or caused byprocessor 2810 from FIG. 22.

Referring to FIG. 18, a display 200 with which a left camera 101 and aright camera 102 are combined, and a 3D transform device 300 of thedisplay 200 are illustrated. The cameras 101 and 102 have a cameracoordinate axis Clc and a camera coordinate axis Crc, respectively. Thedisplay 200 has a display coordinate axis Cp. Referring to FIG. 19, inoperation 2510, a right camera parameter is calculated based on acaptured image. In operation 2520, a left camera parameter is calculatedbased on a captured image. In operation 2530, a first transformationparameter is calculated based on an extrinsic parameter. The extrinsicparameter may include a first extrinsic parameter calculated withrespect to the left camera 101, and a second extrinsic parametercalculated with respect to the right camera 102. The firsttransformation parameter with respect to the first extrinsic parameterand a second transformation parameter with respect to the secondextrinsic parameter may be calculated separately and in accordance withthe operations described above. In operation 2540, the secondtransformation parameter is calculated based on the captured image.Although an example in which two cameras are provided is described,parameters may be estimated using the similar method in a case in whichat least three cameras are provided.

FIGS. 20 and 21 illustrate a process of estimating parameters of adisplay device including a camera according to at least one exampleembodiment. As shown in FIG. 21, a method according to at least oneexample embodiment may include one or more of operations 2710 and 2720.The operations in FIGS. 20 and 21 may be carried out and/or caused byprocessor 2810 from FIG. 22.

Referring to FIG. 20, a camera 103 and a display 203 of a tabletcomputer, a camera 104 and a display 204 of a TV or a monitor, and acamera 105 and a display 205 of a mobile phone are illustrated.Referring to FIG. 21, in operation 2710, a camera parameter iscalculated based on a captured image. In operation 2720, a firsttransformation parameter is calculated based on an extrinsic parameter.Each electronic device may estimate poses of the camera and the displaythereof through the first transformation parameter. The aforementioneddescription may be applicable to an electronic device with which adisplay and a camera are combined, in addition to the tablet computer,the TV, the monitor, and the mobile phone.

FIG. 22 illustrates an electronic system according to at least oneexample embodiment.

Referring to FIG. 22, an electronic system includes a processor 2810, acamera 2820, and a display 2830. The processor 2810, the camera 2820,and the display 2830 may communicate through a bus 2840.

The camera 2820 may capture an image using a well-known method, forexample, a method of transforming an optical image into an electronicsignal. The captured image is output to the processor 2810. Theprocessor 2810 may include at least one device described with referenceto FIGS. 1 through 23, or may perform at least one method described withreference to FIGS. 1 through 23. For example, the processor 2810 mayperform the operations described with reference to FIG. 2. The display2830 may display a first image including a first pattern and a secondpattern.

Although not shown in the drawings, the electronic system may furtherinclude a memory (e.g., as part of the processor 2810 or as a separateelement). The memory may store images captured by the camera 2820,and/or a parameter calculated by the processor 2810. The memory may be avolatile memory or a non-volatile memory.

The processor 2810 may be a special purpose processor that executes aprogram (or computer readable instructions), and controls the electronicsystem. A program code (or computer readable instructions) executed bythe processor 2810 may be stored in the memory. The electronic systemmay be connected to an external device, for example, a personal computeror a network, through an input/output device (not shown), and exchangedata with the external device.

Although FIG. 22 illustrates the processor 2810, the camera 2820 and thedisplay 2830 as being part of a single system, it should be understoodthat each element may exist as a separate device. For example, theprocessor 2810 (including a memory) may exist as a device implemented bya dongle with data interfaces (e.g., universal serial bus (USB)interface, high definition multimedia interface (HDMI), etc.) forsending and receiving data to/from the camera 2820 and the display 2830.

The electronic system may include various electronic systems, forexample, mobile devices such as a mobile phone, a smart phone, apersonal digital assistant (PDA), a tablet computer, and a laptopcomputer, computing devices such as a personal computer, a tabletcomputer, and a netbook computer, and electronic products such as atelevision, a smart television, and a security device for gate control.

The units and/or modules described herein may be implemented usinghardware components and software components. For example, the hardwarecomponents may include microphones, amplifiers, band-pass filters, audioto digital convertors, and processing devices. A processing device maybe implemented using one or more hardware device configured to carry outand/or execute program code by performing arithmetical, logical, andinput/output operations. The processing device(s) may include aprocessor (i.e., a special purpose processor), a controller and anarithmetic logic unit, a digital signal processor, a microcomputer, afield programmable array, a programmable logic unit, a microprocessor orany other device capable of responding to and executing instructions ina defined manner. The processing device may run an operating system (OS)and one or more software applications that run on the OS. The processingdevice also may access, store, manipulate, process, and create data inresponse to execution of the software. For purpose of simplicity, thedescription of a processing device is used as singular; however, oneskilled in the art will appreciated that a processing device may includemultiple processing elements and multiple types of processing elements.For example, a processing device may include multiple processors or aprocessor and a controller. In addition, different processingconfigurations are possible, such a parallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, to independently orcollectively instruct and/or configure the processing device to operateas desired, thereby transforming the processing device into a specialpurpose processor. Software and data may be embodied permanently ortemporarily in any type of machine, component, physical or virtualequipment, computer storage medium or device, or in a propagated signalwave capable of providing instructions or data to or being interpretedby the processing device. The software also may be distributed overnetwork coupled computer systems so that the software is stored andexecuted in a distributed fashion. The software and data may be storedby one or more non-transitory computer readable recording mediums.

The methods according to the above-described example embodiments may berecorded in non-transitory computer-readable media including programinstructions to implement various operations of the above-describedexample embodiments. The media may also include, alone or in combinationwith the program instructions, data files, data structures, and thelike. The program instructions recorded on the media may be thosespecially designed and constructed for the purposes of exampleembodiments, or they may be of the kind well-known and available tothose having skill in the computer software arts. Examples ofnon-transitory computer-readable media include magnetic media such ashard disks, floppy disks, and magnetic tape; optical media such asCD-ROM discs, DVDs, and/or Blue-ray discs; magneto-optical media such asoptical discs; and hardware devices that are specially configured tostore and perform program instructions, such as read-only memory (ROM),random access memory (RAM), flash memory (e.g., USB flash drives, memorycards, memory sticks, etc.), and the like. Examples of programinstructions include both machine code, such as produced by a compiler,and files containing higher level code that may be executed by thecomputer using an interpreter. The above-described devices may beconfigured to act as one or more software modules in order to performthe operations of the above-described example embodiments, or viceversa.

A number of example embodiments have been described above. Nevertheless,it should be understood that various modifications may be made to theseexample embodiments. For example, suitable results may be achieved ifthe described techniques are performed in a different order and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner and/or replaced or supplemented by othercomponents or their equivalents. Accordingly, other implementations arewithin the scope of the following claims.

What is claimed is:
 1. A method of estimating a parameter of a displaydevice, the method comprising: displaying, on a display, a first imagecomprising a first pattern; capturing second images generated when thefirst image is reflected by a reflector at different angles, using acamera combined with the display; and estimating a first transformationparameter between the camera and the display based on the first imageand the second images; and estimating a second transformation parameterbetween the display and an optical layer associated with the displaybased on a second pattern included in the first image and the at leastone of the second images.
 2. The method of claim 1, wherein the firsttransformation parameter comprises at least one of a rotation matrix forperforming a transformation between coordinates of the camera andcoordinates of the display, and a translation vector for performing atransformation between coordinates of the camera and coordinates of thedisplay.
 3. The method of claim 1, wherein the estimating comprisesdetermining the first transformation parameter as a transformationparameter that minimizes projection errors between the first pattern andvirtual patterns corresponding to the second images.
 4. The method ofclaim 3, wherein each of the virtual patterns is a virtual imagegenerated when the first pattern is focused at a corresponding one ofthe different angles in the reflector.
 5. The method of claim 1, whereinthe estimating comprises: estimating geometric relationships between thedisplay and the reflector based on the second images; calculatingprojection errors between the first pattern and virtual patternscorresponding to the second images based on the geometric relationships;and updating the first transformation parameter to reduce the projectionerrors.
 6. The method of claim 1, wherein the first pattern comprises apattern in which a shape is repeated.
 7. The method of claim 1, furthercomprising: estimating an intrinsic parameter of the camera based on thefirst pattern and at least one of the second images.
 8. The method ofclaim 1, wherein the estimating the second transformation parametercomprises estimating the second transformation parameter based on aperiod of the second pattern, a period of a third pattern included inthe at least one of the second images, and a gradient of the thirdpattern.
 9. The method of claim 8, further comprising at least one of:obtaining the period of the second pattern based on a subpixel structureof the display; and obtaining the period of the third pattern and thegradient of the third pattern in a Fourier space in which the thirdpattern is frequency-transformed.
 10. The method of claim 7, wherein theintrinsic parameter comprises at least one of a focal length of thecamera, a center position of the camera, and a skewness of the camera.11. The method of claim 1, wherein the second transformation parametercomprises at least one of a pitch of the optical layer and a rotationangle between the optical layer and the display.
 12. The method of claim1, wherein the second pattern comprises at least one of: a pattern inwhich continuous lines of an identical brightness are arranged atintervals; and a pattern comprising areas with an identical color and anidentical brightness.
 13. The method of claim 1, wherein the secondpattern is included in a shape repeated in the first pattern.
 14. Anon-transitory computer readable medium including computer readableinstructions to cause a computer to perform the method of claim
 1. 15. Athree-dimensional (3D) display device comprising: a camera configured toperform eye tracking for a user; a display configured to display a 3Dimage; and a processor configured to, control the display to display afirst image comprising a first pattern, control the camera to capturesecond images, the second images being generated when the first image isreflected by a reflector at different angles, estimate a firsttransformation parameter between the camera and the display based on thefirst image and the second images, estimate a second transformationparameter between the display and an optical layer associated with thedisplay based on a second pattern included in the first image and atleast one second image of the second images, track eyes of the userbased on the first transformation parameter, and render the 3D imagebased on the tracked eyes.
 16. The device of claim 15, wherein the firsttransformation parameter comprises at least one of a rotation matrix forperforming a transformation between coordinates of the camera andcoordinates of the display, and a translation vector for performing atransformation between coordinates of the camera and coordinates of thedisplay.
 17. The device of claim 15, wherein the processor is configuredto determine the first transformation parameter as a transformationparameter that minimizes projection errors between the first pattern andvirtual patterns corresponding to the second images.
 18. The device ofclaim 15, wherein the processor is configured to estimate geometricrelationships between the display and the reflector based on the secondimages, calculate projection errors between the first pattern andvirtual patterns corresponding to the second images based on thegeometric relationships, and update the first transformation parameterto reduce the projection errors.
 19. The device of claim 15, wherein theprocessor is configured to render the 3D image based on the secondtransformation parameter.
 20. The device of claim 15, wherein the secondtransformation parameter comprises at least one of a pitch of theoptical layer and a rotation angle between the optical layer and thedisplay.
 21. The device of claim 15, wherein the processor is configuredto estimate the second transformation parameter based on a period of thesecond pattern, a period of a third pattern included in the at least onesecond image, and a gradient of the third pattern.
 22. The device ofclaim 15, wherein the processor is configured to estimate an intrinsicparameter of the camera based on the first pattern and at least one ofthe second images, and track the eyes of the user based on the intrinsicparameter and the first transformation parameter.
 23. The device ofclaim 22, wherein the intrinsic parameter comprises at least one of afocal length of the camera, a center position of the camera, and askewness of the camera.
 24. The device of claim 15, wherein the firstpattern comprises a pattern in which a shape is repeated.
 25. The deviceof claim 15, wherein the second pattern comprises at least one of: apattern in which continuous lines of an identical brightness arearranged at intervals; and a pattern comprising areas with an identicalcolor and an identical brightness.
 26. The device of claim 25, whereinthe second pattern is included in a shape repeated in the first pattern.27. A method of estimating a parameter of a display device, the methodcomprising: displaying, on a display, a first image comprising a firstpattern and a second pattern; capturing, a second image generated whenthe first image is reflected in a reflector, using a camera combinedwith the display; estimating a parameter of the camera based on an areacorresponding to the first pattern in the second image; estimating afirst transformation parameter between the camera and the display basedon the area corresponding to the first pattern in the second image; andestimating a second transformation parameter between the display and anoptical layer combined with the display based on an area correspondingto the second pattern in the second image.
 28. The method of claim 27,wherein the parameter of the camera comprises at least one of a focallength of the camera, a center position of the camera, and a skewness ofthe camera.
 29. The method of claim 27, wherein the first transformationparameter comprises at least one of a rotation matrix for performing atransformation between coordinates of the camera and coordinates of thedisplay, and a translation vector for performing a transformationbetween coordinates of the camera and coordinates of the display. 30.The method of claim 27, wherein the second transformation parametercomprises at least one of a pitch of the optical layer and a rotationangle between the optical layer and the display.
 31. The method of claim27, wherein the display device comprises a device configured to play athree-dimensional (3D) image by tracking positions of eyes of a user.32. The method of claim 27, wherein the first pattern comprises apattern in which a shape is repeated.
 33. The method of claim 27,wherein the second pattern comprises at least one of: a pattern in whichcontinuous lines of an identical brightness are arranged at intervals;and a pattern comprising areas with an identical color and an identicalbrightness.
 34. The method of claim 33, wherein the second pattern isincluded in a shape repeated in the first pattern.
 35. A display devicecomprising: a display configured to display a first image comprising afirst pattern and a second pattern; a camera combined with the displayto capture a second image generated when the first image is reflected ina reflector; and a processor configured to, estimate a parameter of thecamera based on an area corresponding to the first pattern in the secondimage, estimate a first transformation parameter between the camera andthe display based on the area corresponding to the first pattern in thesecond image, and estimate a second transformation parameter between thedisplay and an optical layer combined with the display based on an areacorresponding to the second pattern in the second image.
 36. A devicecomprising: a processor; and a memory including computer readableinstructions, which when executed by the processor, cause the processorto, receive a first image displayed by a display, the first imageincluding a first pattern, receive second images, the second imagesbeing reflected versions of the first image that are captured atdifferent angles, using a camera, and estimate a first transformationparameter between the camera and the display based on the first imageand the second images, and estimate a second transformation parameterbetween the display and an optical layer associated with the displaybased on a second pattern included in the first image and the at leastone of the second images.
 37. The device of claim 36, wherein the firsttransformation parameter includes at least one of a rotation matrix anda translation vector, and the memory includes computer readableinstructions to cause the processor to, transform a coordinate system ofthe camera into a coordinate system of the display based on at least oneof the rotation matrix and the translation vector, and render athree-dimensional image using the transformed coordinate system.
 38. Thedevice of claim 36, wherein the processor is configured to estimate bydetermining the first transformation parameter as a transformationparameter that minimizes projection errors between the first pattern andvirtual patterns corresponding to the second images.
 39. The device ofclaim 38, wherein each of the virtual patterns is a virtual imagegenerated when the first pattern is focused at a corresponding one ofthe different angles.
 40. The device of claim 36, wherein the processoris configured to estimate by, estimating geometric relationships betweenthe display and a reflector based on the second images, the reflectorbeing used to generate the second images, calculating projection errorsbetween the first pattern and virtual patterns corresponding to thesecond images based on the geometric relationships, and updating thefirst transformation parameter to reduce the projection errors.
 41. Thedevice of claim 36, wherein the memory includes computer readableinstructions to cause the processor to estimate an intrinsic parameterof the camera based on the first pattern and at least one of the secondimages.
 42. The device of claim 36, wherein the at least one of thesecond images includes a third pattern, and the processor is configuredto estimate the second transformation parameter based on a period of thesecond pattern, a period of the third pattern, and a gradient of thethird pattern.
 43. The device of claim 36, wherein the secondtransformation parameter includes at least one of a pitch of the opticallayer and a rotation angle between the optical layer and the display,and the memory includes computer readable instructions to cause theprocessor to render a three-dimensional image based on at least one ofthe pitch and the rotation angle.
 44. A method comprising: receiving afirst image displayed by a display, the first image including a firstpattern; receiving second images, the second images being reflectedversions of the first image that are captured at different angles, usinga camera; estimating a first transformation parameter between the cameraand the display based on the first image and the second images;estimating a second transformation parameter between the display and anoptical layer associated with the display based on a second patternincluded in the first image and the at least one of the second images;and rendering a three-dimensional image based on the firsttransformation parameter.